[Simulation and Navigation in the Neurosurgical Field Using Three-Dimensional Hholograms by Mixed Reality Devices].

Recently, three-dimensional(3D)holograms from mixed-reality(MR)devices have become available in the medical field. 3D holographic images can provide immersive and intuitive information that has been reported to be very useful for preoperative simulations. Compared with conventional 3D images on a two-dimensional(2D)monitor, 3D holograms offer a higher level of realism, allowing observation of the images anytime and anywhere if the MR device is operational. Even during surgery, surgeons can check realistic 3D holograms in front of them, above the surgical field, without having to turn their heads toward a 2D monitor on the wall. 3D holograms can also be used for neuronavigation if the hologram is tracked to the patient's real head. This method can be defined as 3D augmented reality(AR)navigation, which shows a hologram of a target, such as a tumor or aneurysm, inside the head and brain. In the future, interventions using these techniques with 3D holograms from MR devices are expected to evolve and develop new types of treatments for endoscopic surgery or fluoroscopy-guided endovascular surgery.

Live Cell Imaging by Single-Shot Common-Path Wide Field-of-View Reflective Digital Holographic Microscope.

Quantitative phase imaging by digital holographic microscopy (DHM) is a nondestructive and label-free technique that has been playing an indispensable role in the fields of science, technology, and biomedical imaging. The technique is competent in imaging and analyzing label-free living cells and investigating reflective surfaces. Herein, we introduce a new configuration of a wide field-of-view single-shot common-path off-axis reflective DHM for the quantitative phase imaging of biological cells that leverages several advantages, including being less-vibration sensitive to external perturbations due to its common-path configuration, also being compact in size, simple in optical design, highly stable, and cost-effective. A detailed description of the proposed DHM system, including its optical design, working principle, and capability for phase imaging, is presented. The applications of the proposed system are demonstrated through quantitative phase imaging results obtained from the reflective surface (USAF resolution test target) as well as transparent samples (living plant cells). The proposed system could find its applications in the investigation of several biological specimens and the optical metrology of micro-surfaces.

Comparative oncology chemosensitivity assay for personalized medicine using low-coherence digital holography of dynamic light scattering from cancer biopsies.

Nearly half of cancer patients who receive standard-of-care treatments fail to respond to their first-line chemotherapy, demonstrating the pressing need for improved methods to select personalized cancer therapies. Low-coherence digital holography has the potential to fill this need by performing dynamic contrast OCT on living cancer biopsies treated ex vivo with anti-cancer therapeutics. Fluctuation spectroscopy of dynamic light scattering under conditions of holographic phase stability captures ultra-low Doppler frequency shifts down to 10 mHz caused by light scattering from intracellular motions. In the comparative preclinical/clinical trials presented here, a two-species (human and canine) and two-cancer (esophageal carcinoma and B-cell lymphoma) analysis of spectral phenotypes identifies a set of drug response characteristics that span species and cancer type. Spatial heterogeneity across a centimeter-scale patient biopsy sample is assessed by measuring multiple millimeter-scale sub-samples. Improved predictive performance is achieved for chemoresistance profiling by identifying red-shifted sub-samples that may indicate impaired metabolism and removing them from the prediction analysis. These results show potential for using biodynamic imaging for personalized selection of cancer therapy.

Label-free cell classification in holographic flow cytometry through an unbiased learning strategy.

Nowadays, label-free imaging flow cytometry at the single-cell level is considered the stepforward lab-on-a-chip technology to address challenges in clinical diagnostics, biology, life sciences and healthcare. In this framework, digital holography in microscopy promises to be a powerful imaging modality thanks to its multi-refocusing and label-free quantitative phase imaging capabilities, along with the encoding of the highest information content within the imaged samples. Moreover, the recent achievements of new data analysis tools for cell classification based on deep/machine learning, combined with holographic imaging, are urging these systems toward the effective implementation of point of care devices. However, the generalization capabilities of learning-based models may be limited from biases caused by data obtained from other holographic imaging settings and/or different processing approaches. In this paper, we propose a combination of a Mask R-CNN to detect the cells, a convolutional auto-encoder, used to the image feature extraction and operating on unlabelled data, thus overcoming the bias due to data coming from different experimental settings, and a feedforward neural network for single cell classification, that operates on the above extracted features. We demonstrate the proposed approach in the challenging classification task related to the identification of drug-resistant endometrial cancer cells.

A dataset of digital holograms of normal and thalassemic cells.

Digital holographic microscopy (DHM) is an intriguing medical diagnostic tool due to its label-free and quantitative nature, providing high-contrast images of phase samples. By capturing both intensity and phase information, DHM enables the numerical reconstruction of quantitative phase images. However, the lateral resolution is limited by the diffraction limit, which prompted the recent suggestion of microsphere-assisted DHM to enhance the DHM resolution straightforwardly. The use of such a technique as a medical diagnostic tool requires testing and validation of the proposed assays to prove their feasibility and viability. This paper publishes 760 and 609 microsphere-assisted DHM images of normal and thalassemic red blood cells obtained from a normal and thalassemic male individual, respectively.

Comparison of holotomographic microscopy and coherence-controlled holographic microscopy.

Quantitative phase imaging (QPI) is a powerful tool for label-free visualisation of living cells. Here, we compare two QPI microscopes - the Telight Q-Phase microscope and the Nanolive 3D Cell Explorer-fluo microscope. Both systems provide unbiased information about cell morphology, such as individual cell dry mass, perimeter and area. The Q-Phase microscope uses artefact-free, coherence-controlled holographic imaging technology to visualise cells in real time with minimal phototoxicity. The 3D Cell Explorer-fluo employs laser-based holotomography to reconstruct 3D images of living cells, visualising their internal structures and dynamics. Here, we analysed the strengths and limitations of both microscopes when examining two morphologically distinct cell lines - the cuboidal epithelial MDCK cells which form multicellular clusters and solitary growing Rat2 fibroblasts. We focus mainly on the ability of the devices to generate images suitable for single-cell segmentation by the built-in software, and we discuss the segmentation results and quantitative data generated from the segmented images. We show that both microscopes offer slightly different advantages, and the choice between them depends on the specific requirements and goals of the user.

Fast 2-photon stimulation using holographic patterns.

Two decades after its introduction, optogenetics - a biological technique to control the activity of neurons or other cell types with light - remains a cutting edge and promising tool to study biological processes. Its increasing usage in research varies widely from causally exploring biological mechanisms and neural computations, to neurostimulation and sensory restauration. To stimulate neurons in the brain, a variety of approaches have been developed to generate precise spatiotemporal light patterns. Yet certain constrains still exists in the current optical techniques to activate a neuronal population with both cellular resolution and millisecond precision. Here, we describe an experimental setup allowing to stimulate a few tens of neurons in a plane at sub-millisecond rates using 2-photon activation. A liquid crystal on silicon spatial light modulator (LCoS-SLM) was used to generate spatial patterns in 2 dimensions. The image of the patterns was formed on the plane of a digital micromirror device (DMD) that was used as a fast temporal modulator of each region of interest. Using fluorescent microscopy and patch-clamp recording of neurons in culture expressing the light-gated ion channels, we characterized the temporal and spatial resolution of the microscope. We described the advantages of combining the LCoS-SLM with the DMD to maximize the temporal precision, modulate the illumination amplitude, and reduce background activation. Finally, we showed that this approach can be extended to patterns in 3 dimensions. We concluded that the methodology is well suited to address important questions about the role of temporal information in neuronal coding.

Aberration-free holographic microscope for simultaneous imaging and stimulation of neuronal populations.

A technical challenge in neuroscience is to record and specifically manipulate the activity of neurons in living animals. This can be achieved in some preparations with two-photon calcium imaging and photostimulation. These methods can be extended to three dimensions by holographic light sculpting with spatial light modulators (SLMs). At the same time, performing simultaneous holographic imaging and photostimulation is still cumbersome, requiring two light paths with separate SLMs. Here we present an integrated optical design using a single SLM for simultaneous imaging and photostimulation. Furthermore, we applied axially dependent adaptive optics to make the system aberration-free, and developed software for calibrations and closed-loop neuroscience experiments. Finally, we demonstrate the performance of the system with simultaneous calcium imaging and optogenetics in mouse primary auditory cortex in vivo. Our integrated holographic system could facilitate the systematic investigation of neural circuit function in awake behaving animals.

On-chip label-free cell classification based directly on off-axis holograms and spatial-frequency-invariant deep learning.

We present a rapid label-free imaging flow cytometry and cell classification approach based directly on raw digital holograms. Off-axis holography enables real-time acquisition of cells during rapid flow. However, classification of the cells typically requires reconstruction of their quantitative phase profiles, which is time-consuming. Here, we present a new approach for label-free classification of individual cells based directly on the raw off-axis holographic images, each of which contains the complete complex wavefront (amplitude and quantitative phase profiles) of the cell. To obtain this, we built a convolutional neural network, which is invariant to the spatial frequencies and directions of the interference fringes of the off-axis holograms. We demonstrate the effectiveness of this approach using four types of cancer cells. This approach has the potential to significantly improve both speed and robustness of imaging flow cytometry, enabling real-time label-free classification of individual cells.

Randomness assisted in-line holography with deep learning.

We propose and demonstrate a holographic imaging scheme exploiting random illuminations for recording hologram and then applying numerical reconstruction and twin image removal. We use an in-line holographic geometry to record the hologram in terms of the second-order correlation and apply the numerical approach to reconstruct the recorded hologram. This strategy helps to reconstruct high-quality quantitative images in comparison to the conventional holography where the hologram is recorded in the intensity rather than the second-order intensity correlation. The twin image issue of the in-line holographic scheme is resolved by an unsupervised deep learning based method using an auto-encoder scheme. Proposed learning technique leverages the main characteristic of autoencoders to perform blind single-shot hologram reconstruction, and this does not require a dataset of samples with available ground truth for training and can reconstruct the hologram solely from the captured sample. Experimental results are presented for two objects, and a comparison of the reconstruction quality is given between the conventional inline holography and the one obtained with the proposed technique.

Quantitative phase imaging of living red blood cells combining digital holographic microscopy and deep learning.

Digital holographic microscopy as a non-contacting, non-invasive, and highly accurate measurement technology, is becoming a valuable method for quantitatively investigating cells and tissues. Reconstruction of phases from a digital hologram is a key step in quantitative phase imaging for biological and biomedical research. This study proposes a two-stage deep convolutional neural network named VY-Net, to realize the effective and robust phase reconstruction of living red blood cells. The VY-Net can obtain the phase information of an object directly from a single-shot off-axis digital hologram. We also propose two new indices to evaluate the reconstructed phases. In experiments, the mean of the structural similarity index of reconstructed phases can reach 0.9309, and the mean of the accuracy of reconstructions of reconstructed phases is as high as 91.54%. An unseen phase map of a living human white blood cell is successfully reconstructed by the trained VY-Net, demonstrating its strong generality.

Functionally integrated waveguide illuminator for common-path digital holographic microscopy through random media.

We designed and fabricated a functionally integrated optical waveguide illuminator specially for common-path digital holographic microscopy through random media. The waveguide illuminator creates two point sources with desired phase shifts, which are located close to one another so that the common-path condition of the object and reference illumination is satisfied. Thereby, the proposed device permits phase-shift digital holographic microscopy free from bulky optical elements such as a beam splitter, an objective lens, and a piezoelectric transducer for phase shifting. Using the proposed device, microscopic 3D imaging through a highly heterogeneous double-composite random medium was experimentally demonstrated by means of common-path phase-shift digital holography.

Adaptive constraints by morphological operations for single-shot digital holography.

Digital holography provides access to quantitative measurement of the entire complex field, which is indispensable for the investigation of wave-matter interactions. The emerging iterative phase retrieval approach enables to solve the inverse imaging problem only from the given intensity measurements and physical constraints. However, enforcing imprecise constraints limits the reconstruction accuracy and convergence speed. Here, we propose an advanced iterative phase retrieval framework for single-shot in-line digital holography that incorporates adaptive constraints, which achieves optimized convergence behavior, high-fidelity and twin-image-free reconstruction. In conjunction with morphological operations which can extract the object structure while eliminating the irrelevant part such as artifacts and noise, adaptive constraints allow the support region to be accurately estimated and automatically updated at each iteration. Numerical reconstruction of complex-valued objects and the capability of noise immunity are investigated. The improved reconstruction performance of this approach is experimentally validated. Such flexible and versatile framework has promising applications in biomedicine, X-ray coherent diffractive imaging and wavefront sensing.

Ultrafast light targeting for high-throughput precise control of neuronal networks.

Two-photon, single-cell resolution optogenetics based on holographic light-targeting approaches enables the generation of precise spatiotemporal neuronal activity patterns and thus a broad range of experimental applications, such as high throughput connectivity mapping and probing neural codes for perception. Yet, current holographic approaches limit the resolution for tuning the relative spiking time of distinct cells to a few milliseconds, and the achievable number of targets to 100-200, depending on the working depth. To overcome these limitations and expand the capabilities of single-cell optogenetics, we introduce an ultra-fast sequential light targeting (FLiT) optical configuration based on the rapid switching of a temporally focused beam between holograms at kHz rates. We used FLiT to demonstrate two illumination protocols, termed hybrid- and cyclic-illumination, and achieve sub-millisecond control of sequential neuronal activation and high throughput multicell illumination in vitro (mouse organotypic and acute brain slices) and in vivo (zebrafish larvae and mice), while minimizing light-induced thermal rise. These approaches will be important for experiments that require rapid and precise cell stimulation with defined spatio-temporal activity patterns and optical control of large neuronal ensembles.

AI-based analysis of 3D position and orientation of red blood cells using a digital in-line holographic microscopy.

The morphological and mechanical characteristics of red blood cells (RBCs) largely vary depending on the occurrence of hematologic disorders. Variations in the rheological properties of RBCs affect the dynamic motions of RBCs, especially their rotational behavior. However, conventional techniques for measuring the orientation of biconcave-shaped RBCs still have some technical limitations, including complicated optical setups, complex post data processing, and low throughput. In this study, we propose a novel image-based technique for measuring 3D position and orientation of normal RBCs using digital in-line holographic microscopy (DIHM) and artificial intelligence (AI). Formaldehyde-fixed RBCs are immobilized in coagulated polydimethylsiloxane (PDMS). Holographic images of RBCs positioned at various out-of-plane angles are acquired by precisely manipulating the PDMS-trapped RBC sample attached to a 4-axis optical stage. With the aid of deep learning algorithms for data augmentation and regression analysis, the out-of-plane angle of RBCs is directly predicted from the captured holographic images. The 3D position and in-plane angle of RBCs are acquired by employing numerical reconstruction and ellipse detection methods. Combining these digital image processing techniques, the 3D positional and orientational information of each RBC recorded in a single holographic image is measured within 23.5 and 3.07 s, respectively. The proposed AI-based DIHM technique that can extract the 3D position, orientation, and morphology of individual RBCs would be utilized to analyze the dynamic translational and rotational motions of abnormal RBCs with hematologic disorders in shear flows through further research.

Electrospray ion beam deposition plus low-energy electron holography as a tool for imaging individual biomolecules.

Inline low-energy electron holography (LEEH) in conjunction with sample preparation by electrospray ion beam deposition (ES-IBD) has recently emerged as a promising method for the sub-nanometre-scale single-molecule imaging of biomolecules. The single-molecule nature of the LEEH measurement allows for the mapping of the molecules' conformational space and thus for the imaging of structurally variable biomolecules, thereby providing valuable complementary information to well-established biomolecular structure determination methods. Here, after briefly tracing the development of inline LEEH in bioimaging, we present the state-of-the-art of native ES-IBD + LEEH as a method of single-protein imaging, discuss its applications, specifically regarding the imaging of structurally flexible protein systems and the amplitude and phase information encoded in a low-energy electron hologram, and provide an outlook regarding the considerable possibilities for the future advancement of the approach.

Development of serial X-ray fluorescence holography for radiation-sensitive protein crystals.

X-ray fluorescence holography (XFH) is a powerful atomic resolution technique capable of directly imaging the local atomic structure around atoms of a target element within a material. Although it is theoretically possible to use XFH to study the local structures of metal clusters in large protein crystals, the experiment has proven difficult to perform, especially on radiation-sensitive proteins. Here, the development of serial X-ray fluorescence holography to allow the direct recording of hologram patterns before the onset of radiation damage is reported. By combining a 2D hybrid detector and the serial data collection used in serial protein crystallography, the X-ray fluorescence hologram can be directly recorded in a fraction of the measurement time needed for conventional XFH measurements. This approach was demonstrated by obtaining the Mn Kα hologram pattern from the protein crystal Photosystem II without any X-ray-induced reduction of the Mn clusters. Furthermore, a method to interpret the fluorescence patterns as real-space projections of the atoms surrounding the Mn emitters has been developed, where the surrounding atoms produce large dark dips along the emitter-scatterer bond directions. This new technique paves the way for future experiments on protein crystals that aim to clarify the local atomic structures of their functional metal clusters, and for other related XFH experiments such as valence-selective XFH or time-resolved XFH.

Digital holographic technique based breast cancer detection using transfer learning method.

The digital holographic technique is an interferometric method that provides comprehensive information on morphological traits such as cell layer thickness and shape as well as access to biophysical attributes of cells like refractive index, dry mass, and volume. This method helps characterize sample structures in three dimensions both statically and dynamically, even for transparent objects like living biological cells. This research work captures the digital holograms of breast tissues and analyzes the malignancy of the tissue using a deep learning technique. It enables dynamic measurement of the sample under investigation. Different transfer learning models such as Inception, DenseNet, SqueezeNet, VGG, and ResNet are incorporated in this work. The parameters accuracy, precision, sensitivity, and F1 score of different models are compared and found that the ResNet model outperforms better compared to other models.

Lensless holographic microscope with a time and memory-saving algorithm for large-volume imaging of organoids.

Organoids, the 3D culture systems derived from stem cells, are promising models for human organs. However, organoid study requires large-volume imaging with single cell resolution, which is beyond the spatial bandwidth limit of conventional optical microscopy. Herein, we propose a lensless holographic microscope empowered with a time and memory-saving algorithm. It solves the trade-off between the imaging field of view, resolution, and processing speed, and provides a practical tool for the study of organoids. We first build a compact microscopy system using a multi-angle LED illumination scheme and an on-chip structure. Then we develop a fast angular spectrum formula for fast reconstruction of oblique-illuminated coaxial holography under the under-sampling condition. Additionally, we derive a multi-angle illuminated filtered backpropagation algorithm to achieve high-precision and slice-wise recovery of 3D structures of objects. The reconstruction process demands only 1/50 of the memory required by a traditional optical diffraction tomography algorithm. Experimental results indicate that the proposed method can achieve 6.28 mm × 4.71 mm × 0.37 mm volume imaging within 104 s. Through the standardized polystyrene beads test, we demonstrate that the proposed microscope has micrometer-scale resolution in both lateral and axial directions. In addition, the 3D imaging results of salivary gland organoids show great application prospects of the proposed method in the field of living biological sampling imaging.

Fluholoscopy-Compact and Simple Platform Combining Fluorescence and Holographic Microscopy.

The combination of different imaging modalities into single imaging platforms has a strong potential in biomedical sciences as it permits the analysis of complementary properties of the target sample. Here, we report on an extremely simple, cost-effective, and compact microscope platform for achieving simultaneous fluorescence and quantitative phase imaging modes with the capability of working in a single snapshot. It is based on the use of a single illumination wavelength to both excite the sample's fluorescence and provide coherent illumination for phase imaging. After passing the microscope layout, the two imaging paths are separated using a bandpass filter, and the two imaging modes are simultaneously obtained using two digital cameras. We first present calibration and analysis of both fluorescence and phase imaging modalities working independently and, later on, experimental validation for the proposed common-path dual-mode imaging platform considering static (resolution test targets, fluorescent micro-beads, and water-suspended lab-made cultures) as well as dynamic (flowing fluorescent beads, human sperm cells, and live specimens from lab-made cultures) samples.